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CN ■ Abstract 

We study the problem of detection of a p-dimensional sparse vector of 
■ parameters in the linear regression model with Gaussian noise. We establish 

| the detection boundary, i.e., the necessary and sufficient conditions for the 

possibility of successful detection as both the sample size n and the dimension 
p tend to the infinity. Testing procedures that achieve this boundary are 
, also exhibited. Our results encompass the high-dimensional setting (p>n). 

The main message is that, under some conditions, the detection boundary 
phenomenon that has been proved for the Gaussian sequence model, extends 
to high-dimensional linear regression. Finally, we establish the detection 
boundaries when the variance of the noise is unknown. Interestingly, the 
detection boundaries sometimes depend on the knowledge of the variance in 
j> . a high-dimensional setting. 

o . 

f — . Mathematics Subject Classifications: Primary 62J05, Secondary 62G10, 

62H20, 62G05, 62G08, 62C20, 62G20. 
0> Key Words: High-dimensional regression, detection boundary, sparse vectors, 

^ sparsity, minimax hypothesis testing. 

^ 1 Introduction 

d \ We consider the linear regression model with random design: 

p 

Y i = J2 i X v+& i = l,~,n, (1.1) 
i=i 

where 9j G 1R are unknown coefficients, £j are i.i.d. A/"(0, a 2 ) random variables, Xij 
are random variables, which are identically distributed, and (X^, 1 < % < n) are 
independent for any fixed j with EXy = 0, EXfj = 1. We study separately the 
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settings with known a > (then assuming that a — 1 without loss of generality) 
and unknown a > 0. We also assume that Xij, 1 < j < p, 1 < i < n, are 
independent of £j, 1 < % < n. 

Based on the observations Z = (X, Y) where X = (X^, 1 < j < p, 1 < % < n), 
and Y = (Yi, 1 < i < n), we consider the problem of detecting whether the signal 
9 = (#!, . . . , 9p) is zero (i.e., we observe the pure noise) or 9 is some sparse signal, 
which is sufficiently well separated from 0. Specifically, we state this as a problem 
of testing the hypothesis Hq : 9 = against the alternative 

H Kr : 9 e e k (r) = {9 e IR£ : \\9\\ > r}, 

where denotes the ball in M p of radius k, || • || is the Euclidean norm, and 
r > is a separation constant. 

The smaller is r, the harder is to detect the signal. The question that we 
study here is: What is the detection boundary, i.e., what is the smallest separation 
constant r such that successful detection is still possible? The problem is formalized 
in an asymptotic minimax sense, cf. Section |2] below. This question is closely 
related to the previous work by several authors on detection and classification 
boundaries for the Gaussian sequence model |H El El HDl HU H21 H31 H51 HZl 
HH HH [201 1211 E21 [231 121]. These papers considered model flUE]) with p = n and 
X^ = 5ij, where 5ij is the Kronecker delta, or replications of such a model (in 
classification setting). The main message of the present work is that, under some 
conditions, the detection boundary phenomenon similar to the one discussed in 
those papers, extends to linear regression. Our results cover the high-dimensional 
p^> n setting. 

We now give a brief summary of our findings under the simplifying assumption 
that all the regressors X^ are i.i.d. standard Gaussian. We consider the asymptotic 
setting where p — > oo, n — > oo and k = for some G (0,1). The results 
are different for moderately sparse alternatives (0 < /3 < 1/2) and highly sparse 
alternatives (1/2 < j3 < 1). We show that for moderately sparse alternatives the 
detection boundary is of the order of magnitude 

pi/4 i 

rx^-A^, (1.2) 
whereas for highly sparse alternatives (1/2 < /3 < 1) it is of the order 



^aJ_ (1.3) 

This solves the problem of optimal rate in detection boundary for all the range 
of values (p,n). Furthermore, for highly sparse alternatives under the additional 
assumption 

p l ~ p \og{p) = o{y/n) (1.4) 

we obtain the sharp detection boundary, i.e., not only the rate but also the exact 
constant. This sharp boundary has the form 



J k -^l, (1.5) 
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where 



^V2(1-VT^5), 3/4 </?<!. ' 



The function <£>(■) here is the same as in the above mentioned detection and clas- 
sification problems, as first introduced in [15]. We also provide optimal testing 
procedures. In particular, the sharp boundary (jl.5D - (jl.6p is attained on the Higher 
Criticism statistic. 

One of the applications of this result is related to transmission of signals under 
compressed sensing, cf. [3 Ej. Assume that a sparse high- dimensional signal 6 
is coded using compressed sensing with i.i.d. Gaussian Xij and then transmitted 
through a noisy channel. Observing the noisy outputs Yi, we would like to detect 
whether the signal was indeed transmitted. For example, this is of interest if several 
signals appear in consecutive time slots but some slots contain no signal. Then 
the aim is to detect informative slots. Our detection boundary (11.51) specifies the 
minimal energy of the signal sufficient for detectability. We note that cp(-) < y/2, 
so that successful detection is possible for rather weak signals whose energy is 
under the threshold \Jlk log(p)/n. This can be compared with the asymptotically 
optimal recovery of sparsity pattern (RSP) by the Lasso in the same Gaussian 
model as ours [29], ED] . Observe that the RSP property is stronger than detection 
(i.e., it implies correct detection) but [29] defines the alternative by {9 £ IR^ : 
\6j\ > c^/\og(p)/n,\/j} for some constant c > 2, which is better separated from the 
null than our alternative ©fc(r). Thresholds that are larger in order of magnitude 
are required if one would like to perform detection based on estimation of the values 
of coefficients in the £2 norm [3j [5]. 

In many applications, the variance of the noise £ is unknown. Does the problem 
of detection become more difficult in this case? In order to answer this question, we 
investigate the detection boundaries in the unknown variance setting. Related work 
[27| [28] develop minimax bounds for detection in model (11 .ip under assumptions 
different from ours and under unknown variance. However, [27] does not provide 
a sharp boundary. Here, we prove that for (3 £ (1/2,1) and k\og(p) <C yfn, the 
detection boundaries are the same for known and unknown variance. In contrast, 
when k\og(p) ^> ^/n, the detection boundary is much larger in the case of unknown 
variance than for known variance. We also provide an optimal testing procedure 
for unknown variance. 

After we have obtained our results, we became aware of the interesting parallel 
unpublished work of Arias-Castro et al. [2]. There the authors derive the detection 
boundary in model (11.11) with known variance of the noise for both fixed and 
random design. Their approach based on the analysis of the Higher Criticism 
shares some similarities with our work. When the variables X^ are i.i.d. standard 
normal and the variance is known, we can directly compare our results with [2]. 
In [2] the detection boundaries analogous to (ll.2p and (I1.3P do not contain the 
minimum with the n _1//4 term, because they are proved in a smaller range of values 
(p, n) where this term disappears. In particular, the conditions in [2] exclude 
the high-dimensional case p 3> n. We also note that, due to the constraints on 
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the classes of matrices X, [2] obtains the sharp boundary fll.5p - fll.6p under the 
condition p 1 ~ /3 (log(p)) 2 = o(i/n) which is more restrictive than our condition fll.4p . 
The other difference is that [2] does not treat the case of unknown variance of the 
noise. 

Below we will use the following notation. We write Z = (X, Y) where X = 
(Xij, 1 < j < p, 1 < i < n), and Y = {Yh 1 < i < n) are the observations satisfying 
( II. ip . Let Pg be the probability measure that corresponds to observations Z, P ei 
be those corresponding to observations Zj = (Xn, X ip , Yj) with fixed % = 1, n, 
and Px, Px,i be the probability measures corresponding to observations X or = 
(Xjj, 1 < j < p). We denote by P* and P e x i the conditional distributions of Y 
given X and of Y{ given X^ respectively. The corresponding expectations are 
denoted by Ef and Ef { . Clearly, 

P e (dZ) = P e x {dY)P x {dX), Pe^dZi) = i^(dy)P x>i (dXW), (1.7) 

and 

n 

P e (dZ) = Y[Pe ti (dZ i ). 

i=l 

We denote by Xj e H n the jth column of matrix X = (Xij), and set 

n 

(Xj,Xi) = ^^XjjXu, \\Xj\\ 2 = (Xj,Xj). 

i=l 

2 Detection problem 

For 9 G H p , we denote by M(9) = Yl V j=i ^{e^o} the number of non-zero components 
of 9, where Q{a} is the indicator function. As above, let IR|, 1 < k < p, denote 
the Iq ball in 1R P of radius k, i.e., the subset of 1R P that consists of vectors 9 with 
M(9) < k, or equivalently, 9 G contains no more than k nonzero coordinates. 
In particular IR£ = 1R P . Recall the notation Q k (r) = {9 G 1R£ : \\9\\ > r}. 

We consider the problem of testing the hypothesis Hq : 9 = against the 
alternative Hk, r '■ 9 G ©fc(r). In this paper we study the asymptotic setting where 
p oo, n — y oo and k = The coefficient /3 G [0, 1] is called the sparsity 

index. We assume in this section that a 2 is known. Modifications for the case of 
unknown variance are discussed in Section 14.2.11 

We call a test any measurable function ip(Z) with values in [0, 1]. For a test ip, 
let a(i/}) = Eq{jP) be the type I error probability and (3(ip,9) = Eq{1 — ip) be the 
type II error probability for the alternative 9 G G C IR P . We set 

PU>) = m 6) = sup /3f>, 9), 7 (V0 = 7 (^, ©) = a(V0 + P{1>, ©) • 

We denote by /3(a) = /3 njPi fc(a, r) the minimax type II error probability for a given 
level a G (0, 1), 

(3(a) = inf /3(tjj, 6 fc (r)), < /3(a) < 1 - a . 
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Accordingly, we denote by 7 = / y n ,p,k{f) the minimax total error probability in the 
hypothesis testing problem: 

7 = inf70,6 fc (r)), 

i> 

where the infimum is taken over all tests i/j. Clearly, 

7 = inf (a + /3(a)), < 7 < 1. 

0:6(0,1) 

The aim of this paper is to establish the asymptotic detection boundary, i.e., 
the conditions on the separation constant r = r n p .fc, which delimit the zone where 
ln,p,kij) —> 1 (indistinguishability) from that where ■y n , P ,k( r ) (distinguishabil- 
ity). The distinguishability is equivalent to (3(a) — > 0, V a G (0, 1). We are inter- 
ested in tests ip = ip ntP or if) a = ip n , P ,a such that either 7(1/}) — > or a(ip a ) < a+o(l), 
and f3(ip a ) — > 0. Here and later the limits are taken as p — > 00, n — )• 00 unless 
otherwise stated. 

3 Assumptions on X 

We will use at different instances some of the following conditions on the random 
variables Xij. 

Al. The random variables are uncorrelated, i.e., EX^Xu = for all 1 < j < 
I < p. 

A2. The random variables X^, 1 < j < p, 1 < % < n, are independent. 

A3. The random variables X^, 1 < j < p, 1 < i < n, are i.i.d. standard Gaussian: 

x {j ~jv(o,i). 

Let Uj, 1 < j < p be random variables such that we have equality in distribution 
C(Uj,Ui) = C(Xij, Xu), 1 < j < I < p. We will need the following technical 
assumptions. 

Bl. max« 4 ) = 0(l). (3.1) 

i<j<i<p 

B2. There exists h > such that max^xKp E(exp(hUjUi)) < 00 for \h\ < h , 
and 

log 3 (p) = o(n). (3.2) 
B3. There exists m G IN such that maxi<j<;< p E( \UjUi | m ) < oo ; and 

log 2 (p)p 4/m = o(n). (3.3) 

Assumption Bl implies that 

max E(\UjUi\ m ) = 0(1), m = 2,3,4. (3.4) 

i<j</<p 
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In particular, Assumption Bl holds true under A2 if 



max E(Uf) = 0(1). (3.5) 



i<i<P 



If (XijXu, i — 1, . . . , n) are independent zero-mean random variables, we have (cf. 
[25], p. 79): 

n 

E\(X j ,X l )\ m < Cim^-^EQXijXun, m>2. 



i=l 

This and ([32]) yield 

£ E(\(X j ,X l )\ m ) = 0(n m / 2 p 2 ), m = 2,3,4. (3.6) 

i<i<^<p 

Finally, Assumptions Bl and B2 hold true under A3 and (13.21) . 

4 Main results 

4.1 Detection boundary under known variance 

For this case we suppose a = 1 without loss of generality. 

4.1.1 Lower bounds 

We first present the lower bounds on the detection error, i.e., the indistinguishabil- 
ity conditions. We assume that k = p 1- ^, /3 G (0, 1). Indistinguishability conditions 
consist of two joint conditions on the radius r = r np . The first one is 

rl p = o^" 1 / 2 ). (4.1) 

The second condition differs according to whether /3<l/2or/3>l/2. If /3 < 1/2 
(i.e. p = 0(k 2 )), which corresponds to moderate sparsity, we require that 

r\ p = o{^/n) . (4.2) 

The case (3 > 1/2 (i.e. k 2 = o{p)) corresponds to high sparsity. We define x np by 
r np = Xn, P \J k\og{p) / n. Then, we require that 

limsup(x niP - <p(f3)) < 0, (4.3) 

where <p((3) is defined in (jl.6p . Clearly, condition (14. 3 p implies r 2 p = 0(k\og(p)/n), 
which is stronger than (14. 2 p when /3 > 1/2. 

Theorem 4.1 Assume Al, Bl, k = p 1 ^ 13 and either B2 or B3. We also require 
that r np satisfies glty and either (gj|) (for /3 e (0, 1/2]) or gjfy (for (3 G (1/2, 1) ). 
Then, asymptotic distinguishability is impossible, i.e., 'Jn,p,k( r n P ) ~^ 1- 
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Remark 4.1 This theorem can be extended to non-random design matrix X . In- 
spection of the proof shows that, instead o/Bl, we only need the assumption: For 
some B n}P tending to oo slowly enough, 

|(X i ,X,)r<S„ tP n m /V > m = 2,3,4. (4.4) 

i<i<'<p 

Indeed, Bl used in the proofs only to assure that lj4-4\ ) holds true with Px- 
probability tending to 1 (this is deduced from assumption Bl and A3. 6]) ). 

Also instead of B2 and B3, we can assume that there exists rj ntP — > such that 

r n P max \(Xj,Xi)\ < r) n , p k, max IHXJ 2 - n\ < r] ntP n. (4.5) 
i<j<i<p i<j<p 

Under B2, B3, relations (f^.5| ) ZioZd u>zt/j Px -probability tending to 1, see Corollary 

The result of the theorem remains valid for non-random matrices X satisfying 
( O and (f^l ). 

4.1.2 Upper bounds 

In order to construct a test procedure that achieves the detection boundary, we 
combine several tests. 

First, we study the widest non-sparse case k = p, i.e., we consider O p (r) = {0 G 
IR P : ||0|| > r}. Consider the statistic 

n 
i=l 

which is the i?o _cen tered and normalized version of the classical x^-statistic 
Sr=i ■ The corresponding tests ipa an d V ; ° are °f t ne form: 

where a G (0, 1), u a is the (1 — a)-quantile of the standard Gaussian distribution 
and T np is any sequence satisfying T np — > oo. 

Theorem 4.2 For all a G (0, 1), we have: 

(i) Type I errors satisfy a(ip^,) = a + o(l) and a(i/j°) = o(l). 

(ii) Type II errors. Assume A2 and Bl and consider a radius r np such that 
nr n P ~^ 00 ■ Then, we have /3(^°, 9 p (r np )) — > 0. If T np is chosen such that 
lim sup T np n- l ' 2 r- 2 < 1, then (3(ip°,Q p (r np )) 0. 

Recall that we can replace Bl by (13. 5p under A2. If nr^ — > oo, then one can take 
T np such that 7 n p ('?/' , Q p (r np )) — > under A2, Bl. This upper bound corresponds 
to the part ( 14. ip of the detection boundary. 
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We now introduce a test ^\ that achieves the second boundary (14. 2p . Consider 
the following kernel 

v 

K(Z i} Z k ) = p-^YiYu XiiXkr 

The [/-statistic t\ based on the kernel K is defined by 

t^N- 1 ' 2 K ( Z ^ Z k), N = n(n -l)/2. 

l<i<k<n 

Note that the [/-statistic t\ can be viewed as the ifo-centered and normalized 
version of the statistic Xp = n Yl V j=i @] based on the estimators 9j = n' 1 Y17=i ^X^: 

p p n 

j=l l<i<k<n j=l i=l 

Indeed, up to a normalization, the first sum is the [/-statistic t±, and moving off 
the second sum corresponds to centering. 

Given a G (0, 1), we consider the test ip\ = ~R tl >u a - 

Theorem 4.3 Assume A2 and Bl. For all a G (0, 1) we have: 

(i) Type I error satisfies: a{ipa) = a + o(l). 

(ii) Type II errors. Assume that p = o(n 2 ) and consider a radius r np such that 
nr lp/y/P^ oo. Then, P(ipl,,G p (r np )) -> 0. 

Remark 4.2 Combining the tests ipa/2 an d we obtain the test = 

max(^, 2 ,^L 2 ) of asymptotic level not more than a. Moreover, it achieves 
(3(ipa, Q p (r np )) — > for any radius r np satisfying 



We can omit the condition p = o(n 2 ) since the test ip^/ 2 achieves the optimal 
rate for p > n. Combining this bound with Theorem J^.l\ we conclude that i\)* a 
simultaneously achieves the optimal detection rate for all (5 G (0, 1/2]. 

We now turn to testing in the highly-sparse case /3 G (1/2, 1). Here we use a 
version of "Higher Criticism Tests" (HC-tests, cf. [8]). Set 

y i = {Y,x i )/\\Y\\, \<i<p. 

Let qi = P(|jV(0, 1)| > \yi\) be the p- value of the i-th component and let denote 
these quantities sorted in increasing order. We define the HC-statistic by 

t HC = max v = • (4.7) 

h ?«)<i/2 y/q^l - g w ) 

Given a constant a > 0, the HC-test ip HC rejects Hq when the statistic tnc is larger 
than (1 + a)y/2 log log p. 
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Remark 4.3 The cutoff' 1/2 in the definition ( |^. 7| ) oftnc can be replaced by any 

ce (0,1). 



Theorem 4.4 Assume A3 (Xij are i.i.d. standard Gaussian). 

(i) Type I error satisfies a(ip HC ) = o(l). 

(ii) Type II error. Consider k = p x ~^ with (3 £ (1/2,1) and assume that 
k\og(p) = o{n). Take a radius r np = x np y / k\og(p)/n such that liminf(x np — 
(p{p)) > 0. Then, we have ^ HC Q k (r np )) -»■ 0. 

Remark 4.4 If klog(p) = o(n), the HC-test asymptotically detects any k-sparse 
signal whose rescaled intensity r np ^n/(k\og(p)) is above the detection boundary 

m- 

Remark 4.5 Assume A3. Combining the tests and ip HC , we obtain the test 
ipa HC = max(ilj^,ilj HC ) of asymptotic level not more than a. Moreover, it achieves 
fii^lti Qk( r np)) for any radius r n ^ p satisfying 

liminf x U)P > tp{p) or r 2 np > —= . 

\/n 



We can omit the condition k log(p) = o(n) since the test ip^ achieves the optimal 



rate for fclog(p) ^> \Jn. Combining this bound with Theorem 4-f we conclude that 
^a HC simultaneously achieves the optimal detection rate for all f3 £ (1/2, 1). 

In conclusion, under Assumption A3, the test max(ip^ 2 , ip^ 2 , ip HC ) simultane- 
ously achieves the optimal detection rate for all (3 £ (0, 1). The detection boundary 
is of the order of magnitude 



A; logo 1 

— A ^77 • 4.8 

Furthermore, we establish the sharp detection boundary (i.e., with exact asymp- 
totic constant) of the form 

Iklogp 
V n 

for (3 > 1/2 and k\og(p) = p 1_/3 log(p) = o(y/n). 

4.2 Detection boundary under unknown variance 
4.2.1 Detection problem 

Since the variance of the noise is now assumed to be unknown, the tests ip under 
study should not require the knowledge of a 2 . The type I error probability is now 
taken uniformly over a > 0: 

a w tyO=sup£ 0l «r(VO • 

<7>0 
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The type II error probability over an alternative 6 C f p is 

/3 m (^,0) = sup ^,9a,a) = sup EeaA 1 -^)- ( 4 - 9 ) 
6>ee,o->o eee.oo 

Similarly to the setting with known variance, we consider the sum of the two errors: 

7 «"(^ 0) = a un {^) + (3 un {^, 6). 

Finally, the minimax total error probability in the hypothesis testing problem with 
unknown variance is 

7r M (r) = inf 7 m (^,e fc (r)) 

4.2.2 Lower bounds 



Take r np = x n ^\J k\og(p) / n. As in the case of known variance, we consider the 
condition 

limsup(a; niP - <p(P)) < . (4.10) 
Theorem 4.5 Fix some (3 > 1/2 and assume A3. If Condition d4.10\ ) holds and 



if k\og(p) = o(n), then distinguishability is impossible, i.e., 7^pfc(?"n.p) 1 • 

If klog(p)/n — > oo, then for any radius r > 0, distinguishability is impossible, 

7n^,fc( r ) -> !• 

The detection boundary stated in Theorem 14 . 51 does not depend on the unknown 
a 2 . This is due to the definition (14. 9p of the type II error probability (3 un (i/j, 0fc(r)) 
that considers alternatives of the form a8 with 9 e Ofc(r). 

4.2.3 Upper bounds 

The HC-test ip HC defined in (14. 7p still achieves the optimal detection rate when 
the variance is unknown as shown by the next proposition. 

Proposition 4.6 Assume A3 (Xij are i.i.d. standard Gaussian) . 
(i) Type I error satisfies a un (tp HC ) = o(l). 

(i%) Type II error. Consider k = p l ~^ with ft G (1/2,1) and assume that 
Hog(p) = o{n). Take a radius r np = x np ^k log(p)/n such that liminf(x np — 
ip{fi)) > 0. Then, we have (3 un (ifj HC , fe (r np )) ->■ 0. 

In conclusion, in the setting with unknown variance we prove that the sharp 
detection boundary (i.e., with exact asymptotic constant) of the form 



khgp 

holds for > 1/2 and k\og(p) = p 1 ' 13 \og(p) = o(n), i.e., for a larger zone of values 
(p, n) than for the case of known variance. However, this extension corresponds 
to (p, n) for which the rate itself is strictly slower than under the known variance. 
Indeed, if the variance a 2 is known, as shown in Section 14.11 the detection bound- 
ary is of the order ( 14. 81) . Thus, there is an asymptotic difference in the order of 
magnitude of the two detection boundaries for fclog(p) ^> y/n. 
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5 Proofs of the lower bounds 



5.1 The prior 

Take c G (0,1), h = ck/p, b = r np /cyk, a = by/n. Note that the condition 
r n P = ^l/v 7 ^) i s equivalent to b A k 2 n = o(l). 

Let us consider a random vector 9 = (9j) with coordinates 

9j = bej, where Sj G (0, +1, — 1) iid, 

such that 

Prob(ej = 0) = 1 - h, Prob(ej = +1) = Prob{ej = -1) = h/2. 

This introduces a prior probability measure ttj on 9j and the product prior measure 
7r = Y\j=i 7r 3 on ^- The corresponding expectation and variance operators will be 
denoted by E n and Vaiv 

Lemma 5.1 Let k — > oo. Then 

tt(9 g IR|, \\9\\ > r) ->• 1. 

Proof. Observe that 



3=1 3=1 



V 



We have 
and 



E n (\\9\\ 2 ) = b 2 ph = r 2 /c, E n (m(9)) = ph = ck, 



Var w (||#|| 2 ) < phb A = r 4 /(fcc 3 ), Var^(m(#)) < ph = ck. 
Applying the Chebyshev inequality, we get with C = c _1 > 1, 

n(\\9\\ 2 < r 2 ) = n(EAM 2 ) - \\9\\ 2 > r\C - 1)) < -+ 0, 

and similarly, ir(m(9) > k) — > 0. □ 

Lemma 15.11 implies that, in order to obtain asymptotic lower bounds for the 
minimax problem, we only have to study the Bayesian problem which corresponds 
to the prior tt, see for instance [IE], Proposition 2.9. Consider the mixture 



P w (dZ) = E n P e (dZ) = [ P e (dZ)n(d8) 

Jw 



and the likelihood ratio 
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In order to prove the lower bounds we only need to check that 

L n (Z) —7-1 in P — probability. (5.1) 

Consider x = limsupa; n) p. If (3 < 1/2, then x = since nb 2 = 0(1). For 
f3 > 1/2, we take c G (0,1) such that x c = x/c < <p(f3), which is possible as 
x < <p(/3). We will use the short notation x and a for x c and a c = by/n = a/c. We 
set 



a, = bWXjl y) = {Xj, Y)/\\Xj\\, x 3 = aj /y/log(p), Tj = a 3 /2 + log^" 1 )/ 



which corresponds to he a j +a 3 T = 1. 

5.2 Study of the likelihood ratio L n 

First observe that by (11.71) 

/ dP x 

P n (dZ) = P x (dX)E 7T {P e x (dY)) , L n (Z) = E n (jj5Jc(Y) 

Note that conditional measure P x corresponds to observation of the Gaussian 
vector J\f(v,I n ) where v = Y7j=i®jXj, I n is the n x n identity matrix, and the 
likelihood ratio under the expectation is 

dP x 

—^—(Y) = exp(— ||f || 2 /2 + (v, Y)) = gg(Z)e~ A ( x,8 \ 



dP X 



where 



g e (Z) = l[exp(-e]\\X J \\ 2 /2 + e j (X J ,Y)), A(X,0)=2 ^ 

3=1 l<j<'<P 

(5.2) 

A(Z) = E v (g g (Z)) = f[(l -h + he- b2 ^ 2 / 2 cosh(b(X v Y))) . 

3=1 

We define rjj = e -*\\Xj\\ 2 /2 cos h(&(Xj, Y)) - 1. Take now 5 > and introduce the 
set 

S x = {9 e W : | A(X, 0)| < 5}. 

We can write 

L n (Z) = [ g e (Z)e- A ^TT(d6) > e~ s [ g e {Z)*(dd) = e^A(Z)vr z (S x ), 

J MP JT, X 

where tiz = YHj=i is the random probability measure on H p with the density 

12 



i.e., the measure n Z j is supported at the points {0, b, —b} and 

l-h h%. 



1 + H .' 2 , i + 

where we set 

4 = ~ a V 2 ± a iy'i > Vj = ~Y + ~7~ ~ L 
Proposition 5.1 In Po-probability , 

Tizi^x) -> 1. (5.3) 
Proof of Proposition [BTTl is given in Section I5TB1 
Proposition 5.2 In Po-probability, 

A(Z) -> 1. (5.4) 
Proof of Proposition I5T21 is given in Section l5~4l 

Propositions 15.11 and 15.21 imply that, for any 5 > 0, 

P (Z:L W (Z)> 

Since -Eo-^tt = 1 arid L 1X {Z) > 0, this yields L n — > 1 in Po-probability. This yields 
indistinguishability in the problem. □ 

5.3 Proof of Proposition 15.11 
5.3.1 Replacing the measure ttz by ttz 

Let us consider the random measure tt z = Yl^=i^z,j, where n Z j is supported at 
the points {0, b, —b} and 

~ m\ i Qz>3 ~ 1z,j 

n ZJ (0) = 1 " 2 ~ 2 ' nz >s( ±b ) = 2 ' 

where 

? ± . = (V2)e^% ± , ^ = {/^ < 1} = {± y ; < Tj} 

and observe that the event Af implies q% ■ < 1/2, i.e, the measures n z ,j are cor- 
rectly defined. We define the event A = A np = n^ =1 (Aj fl Aj). 

Lemma 5.2 

Po(A n , p ) -»• 1. 
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Proof. Denote A c the complement of the event A. Since y'j ~ A/"(0, 1) under P , 
we have 

P X ((X, P ) C ) < $>o X ((A + ) C ) + ^o X ((A 7 ) C ) = 2 E $ (- T ,)- 
j=i i=i 

By Corollary 17.11 we get aj = b\\Xj\\ ~ &a/ti uniformly in 1 < j < p in P_x- 
probability. By (17. ip this implies $3=i^K — ^j) = i n Px-probability. □ 

We can replace the measure 7r^ by ff^ in ( 15. 3p . This follows from the following 
lemma 

Lemma 5.3 In Po-probability, 

E iz \dn z /dn z - 1| -»■ 0. (5.5) 

Proof. Applying the equality E^ z (dn z / 'dn z ) = 1 an d the inequality 1 + x < e x , 

we get 

(E iz \dn z /dn z - 1|) 2 < P iz (c/7r z /^-l) 2 = P^(c/vr z /^ z ) 2 -l 

p 

= n ( dnz * i djxz j ) 2 ~ 1 

p 



^(l + ^dTrzj/tiTfzj-l) 2 )-! 



p 



- exp \^2 E ^A d7r z,j/dTtz,j - 1) 2 J - 1. 

Consequently, we only have to prove that in P -probability, 

p 

H(Z) = ^Ek^ickzj/ditzj - I) 2 0. 
i=i 

Since H(Z) > 0, the last relation follows from 

Eq(H) ->■ 0, in Px-probability 
by Markov inequality. Observe that 

E %7 Ad-Kz i/d-Kz 1) = ^— r — iL - ; ^ : 

By Lemma 15.2^ it is sufficient to study these terms under the event A which cor- 
responds to max!<j<p q z - < 1/2. Under this event, we have h^j = Qzj/^ji = 
1 + Q z j + Qz j ~ h, and direct calculation gives 

(htj ~ Qh? , (h Ztj ~ Qz,? , {h+j + h- ZJ - q+ tj - q^f _ (q+ d + <^)A 2 



2 «zj 2 ^i 2 (2 -Qh-Gj) ^-Qtj-Qz,) 
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where 

A, = qtj + qz, —h — h(e d H A + + e d J 1 AJ - 2)/2. 
Since maxi<j<pq Z j < 1/2, we only have to control the sum J^ =1 A 2 . 

E*1L A A 2 < y ^e a ?$(Tj - 2a,) + e" a ? - 4$(T i - a 3 ) + 2 

CASE 1: nb 2 = 0(1). By corollary EU (aj/i^/nb) - 1) = o Pjt (l). Consequently, 
$(T i -2a J ) = l-o Px (p" 2 ). 



A' 



2 



< ^! s inh 2 (n6 2 (l+o Px (l))/2)+ OPx (l) 



ph 2 nb 2 



+o Px (l) = opJV 



3 r ;, P = °(y/p/ n )- 



CASE 2: limsuprio 2 = oo. This implies that k 2 = o(p) and therefore ph 2 = o(l). 



^E A 



P^ e nfc2(l +0px (l)) + ^ = ^ {2 p. 1)+x 2 +0px{1) + ^ _ 



Since a; < y?(/3) < \/2/3 — 1 for (3 > 1/2, this allows to conclude. □ 



5.3.2 Study of E* Z A 



By Lemma [5.2[ the relation (I5.3P follows from 7r(S) — > 1, in P -P r °bability. Thus, 
we only need to check that in P -probability, P^- Z A 2 — > for A = A(X, 6) defined 
by (15. 2p . By Markov inequality, the last relation follows from 

E*(E iz A 2 ))) -> 0, in P x -probability. 
Let us introduce the events X n ,p- Taking a positive family r\ = r] n p — > 0, we set 
X^KWX^-^Krjn}, X" = {log(p) | (Xj, X t ) | <r] n}, X n>p = f| (A? n AT*') 

It follows from Corollary 17.11 that, under assumptions B2 or B3 we can take rj = 
Vn, P — > such that Px{X n ,p) ~^ 1- We have 

v 



E iz A 2 


= b 4 E 


where 






A 2 




A 3 




A, 



E 9ji 9j2 9ja () r, ( X ji ' X h ) ( X 33 ' X . 



2A 2 + 6A 3 + 24A 4 , 



&4 E ^(ii)^-^) 2 ' ( 5 - 6 ) 

l<jl<?2<P 

6 4 E ^ ( e ii e J2 e js) POi^.72) (-Xju^ja)) ( 5 - 7 ) 

l<?'l<j2<73<P 

6 4 E ( e h € h € j3 e n) ( X jl1 X j2) ( X j31 X ji)- ( 5 - 8 ) 

l<jl<32<h<34<P 
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5.3.3 Expectation over jtz and over 

Let us define the variables 77^ in {1,-1}. The expectations over 7r^ are of the form 

Biz( 44) = (^±^M+fe) = i E ^ e (59) 

7j 1; r? 2 fe=l 

VltV2,V3 k=l 
1 4 1 4 

fiii^t -9 ifc ) =^ ^w^nc- ( 5 - n ) 



16 1AVlft JJfc/ 16 

fc=l Vi,V2,iT.l,V4 k=l 



Let us take the expectation .E^ over F of each of these expressions. We define 
the vector V = bY^k = iVkXj k . Here, £y refers to the expectation of Y over the 
Gaussian measure Af(V,I n ). We derive that 



E * ( ] \qf u \ = ^E* ( e -JE£= 1 6 a II^JlW,Er=i^) Jj n 



/it 



2^ 

vfc=l / \ fc=l 

.IIIT/-II2. 



{Cy,TffcX ifc )<T J - fc ||X ifc | 



exp I * £ **) ) * ( e-^HY.V) JJ n {(y ,„ Xjt)<TjJ ^ ||} 

\ l<r<s<m / \ fe=l 

h m ( \ ( m 

— exp 6 2 £ 77,77, (X jV , X js ) J] n {( y i%Xjfc )<T . fc , 

\ l<r<s<m / \fe=l 



2 

where 



— exp ( b 2 VrVs (X jr , X js ) ) P h ,..., jm (77) , 



l<r<s<m 



,-..('/) = *v 



{(Y,mX jk )<T jk \\x jk \\} 



K k=l 



e ° n 
n 

fc=i 



^ II" 



{»»Vfc <T >fc k )/\\ x j k 11} 

fc=l 



Let us define 



m h (v)=Vk Yl {X js ,X jk )/\\ X jk I), z k = r] k y' jk . 

8=1, Sy^fe 
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Then, Pj u ...j m {7]) writes as 

!). j, ('/) = Pq (zi < T h - a h - bm h {ri) ...,z m <T jm - a jm - bm jm (rj)) 

We have 



2 



E o X (]IC) = ^exp( VlV2 b 2 (X n ,X j2 ))P n ,Mi (5-12) 



vfc=l 
3 



^(IIC) = Y eXP \ b2 E VsVr(X^X lr ))Pn,h,rM, (5.13) 

Ufc=l j \ Ks<r<3 



1 



^(n^l = T^ ex P( 62 E iMXi.,**))^**^)- (5-14) 
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Jfc=l / \ Ks<r<4 



5.3.4 Evaluation of probabilities Pj 1 ,...,j rn (v) 
By definition of (zi, . . . , z m ) we have 



E*z k = 0, E*4 = l, E*z k z s = r ks (v)= T| y i]W if ' 1 < ^ < s < m. 



Denote T Jfe = T Jfe — a Jfc . Observe that 



,.('/) = 

k=l 

+ °( E ^.(n)(-4-^i*W.-4-^.W)). (5-15) 

\l<fc<s<m / 

where we set, for the Gaussian random vector (z\, z 2 ) with Ez k = 0, Ez\ = 1, k — 
1,2, Ez x z 2 = r, 

P r (t l7 t 2 ) = P{Z X < h, Z 2 < t 2 ) = P(Z! > ~h, Z 2 > -t 2 ). 

The control of -Pj 1 ,... Jm ( ? 7) then depends on the sequence x np . 

CASE 1: x — 0. Under the event X n>p , we have max,,- aj = o(W\og(p)) and 



Pjkl V 1°s(p) ~~ °°. Under the event X np , we have 

b\m jk ( V )\ < bJ2 \(X js ,X jk )\/\\X jk \\ < (6Vn/log(p)) = o(1/VMp)) 

It follows that 

max$(-T ifc - bm jk (r])) = o(p~ a ), V a > 0. 
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We conclude 

,, ('/) = 1-0 [ - bm jk (rj)) )=1- o{p-< 



CASE 2: x > 0. We have under the event X UtP , b\mjk(r))\ = o{by/n/ log(p)) = o(l) 
and Tj k b = O (log(p) / y/n) . Hence, Tj k b\mj k (r))\ = o(l). Applying Lemma I7T2| we 
bound the first term in (15.151) 

m mm 

J2$(-f jk -bm jk (r,)) = X>(-4) -bJ2 m i^M-T jk ) 

k=l k=l 

m 



k=l 



k=l 



Let us define 



fl m =x;*(-4) = o(w 1 : 



(5.17) 



k=l 



by (HI]) and (Q since x < <p{/3) < y/2(l - VT^). Applying again (17. ip and 
\2\i. we get 



b(X js ,X jk )<S>(-f jk )/\\X 



.Ik 



o 
o 



T jk {X js ,X jk m-f jk )n- x _ 
T jk ^{-f jk )\r ks \\ =o(\r ks \/(ph)) 



b\X^X 3k )% k $>{-f Jk )n- 1 
It follows that 



0(T? k M-f jk )rl = o{rlJ{ph)). 



J2$(-f jk -bm jk ( V )) = R m - Yl °(\rks\/(ph))+ £ o(r 2 ks /(ph)) 

l<k<s<m l<k 

= E o^ + lrksl + rU/iph)) 



fe=i 



l<fc<s<m 



l<fc<s<m 

Let us turn to the second term in ( 15. 15ft . If Tj k > log(p), then 



Tj k ~ bm jk (v)i ~Tj. - bm js (r})) < $(-f jk - bm jk (rj)) = o(((ph) 2 ) 



If Tj k < log(p), we have Tj k r ks — o(l) under the event X np . By Lemma 17731 and 
previous evaluations, we get 



-T jk - bm jk (ri), -T js - bm js (r]) ^ 
= $(-f Jfc - bm jk {r,)) <S>(-f js - bm js {ri))0 (l + r 2 ks + \r ks \) = o ((ph)~ 2 ) . 
Finally, we obtain 



l\r ,,-('/) 



1-Rm + ol E \ r ks\/ph\ 
\l<k<s<m / 



o{{ph)~ 



l + odph)- 1 ). 



(5.18) 
(5.19) 
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5.3.5 Evaluation of A 2 

We have b 2 ra.ax. 1 <j 1< j 2 < v \(Xj 1 , Xj 2 )\ = o(l) under the event X n<p . Since Pj^irj) 
0(1), we get from ([532]) 

By Assumption Bl, we have 



sup ^(I^I^^OH . 
It then follows from (I5.6P and (15.91) that A 2 is of the order 



b 4 h 2 Yl ( X hi X h? x P 2 h 2 b 4 n x nk 2 b 4 -> 

i<ii<i2<p 



in Px-P r obability. 



5.3.6 Evaluation of A* 



Let us turn to A 3 . Consider % as independent random variables taking values in 
{— 1, 1} with probabilities 1/2. By (1 5 . 1 [) and (15.131) . we can write 

E iz {()],(> j J',,) = (wsexp (b 2 VsVr(X js ,X jr )) P jljj2>j3 (r)) 

V \ l<s<r<3 / / 

Under the event X n ^ v it follows from (I5.16p . (I5.19p . and the definition of X n ^ v that 
P h,h&M = 1 + o^phy 1 ) and b 2 ^ r] s r] r (X js , X jr ) = o(l). 

l<s<r<3 

It follows that 

^,KMa) = y^U^expU 2 £ TiMXj.,X»)))+o(h a p- 1 ). 

V \ l<s<r<3 / / 

By Taylor expansion of the exponential function, the expectation over rj is of the 
form, for c sr = b 2 (X js ,X jr ), 

E v (??2?73 (l + ^1^2Cl2 + ?7l??3Cl3 + ^2^23 + O (c\ 2 + cj 3 + C$ 3 ))) 

= b 2 {x^x n ) + oU (x js ,x jr )A. 

\ l<s<r<3 / 



Under the event X n>p , we derive from (15 ,7p that 



A 3 < h 3 (& 6 0(i?!) + b 8 0(H 2 )) + b^H^p- 1 ] 
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where 



Hi = ^2 \( X h' X h)\\( X h' X h)\\( X h' X h)\-> 
i<ii<j2<i3<p 

H2 = £ I ( X h , X h )\\{ X 3l, X 33 ) I ( X Js , X jr ) 2 , 

l<ii<i2<J3<P l<s<r<3 
#3 = £ I fe, ^2) 1 1 (^1,^3) I" 

l<jl<J2<J3<P 



Since 



1(^,^)11(^,^3)11(^,^)1 < \(X n ,X n )\ 3 + \(X n ,X J3 )\ 3 + \(X Jr ,X Ja 

\( X 3i> X h ) 1 1 ( X h > ^73 ) I (^72 > ) 2 - (^j'i j -^j'a ) 4 + (-^ii ' X h ) 4 + ( X h j -^'3 ) 4 j 

| (X^ , Xj 2 ) 1 1 (Xji , ) 1 < (X^ , X i2 ) 2 + (X jl , Xjg ) 2 , 
we derive from (13.61) 

E X E X = 0{p 3 n 3 ' 2 ), E X H 2 = 0{p 3 n 2 ), E X E X = 0(p 3 n). 
Applying Markov's inequality yields 

H x = Px (p 3 n 3/2 ), H 2 = Px (p 3 n 2 ), E x = Px (p 3 n). 
Combining these bounds, we obtain 

A 3 = Px ((b 6 h 3 p 3 n 3 ^ 2 ) + Px (b 8 h 3 p 3 n 2 ) + b 4 o Px (b 4 h 2 - 2 
Since b 4 k 2 n = o(l), hp x k, b = o(l), we get A 3 = o Px (l). 

5.3.7 Evaluation of At 

Let us evaluate the item A4. Similarly to A3, we can write 

E *z ^'r rAi: ( 'r, ) = VWVW exp & 2 £ T] s rj r (X ja , X jr ) J P, 

V \ l<s<r<4 / 

Under the event we have 

b 2 vMXj.M = o(i). 

l<s<r<4 

CASE 1: x > 0. By fl5TT8|) . we have 



31 ,32 ,33,34 



(17) 15.20) 



p. . . . 

- 1 31,32,33,34 



(r]) = l-R 4 + ol £ |r sr |//ip j +o((p/i)" 

\l<s<r<4 / 
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Applying a Taylor expansion of the exponential term in ( I5.20p yields 
E v V1V2V3V4 exp b 2 ^ VsVr(X js ,X jr ) 

to) 

\ \ l<s<r<4 / / 

= U 2 E (i-^i) +o(l«il) + o(N) 

\ \ l<s<r<4 / / 



= 0(1^1) + 0(\5 2 \), 
where 



51 = OU MM 2 ), 

\ l<s<r<A J 

5 2 = o I ^ \ r ks\/ph J + o((ph)~ 

\l<k<s<4 / 



CASE 2: x = 0. By (|5.16p . Pj 1 ,j 2 ,j3,j4( 7 l) = 1 - o(p 2 ). Arguing as in Case 1, we 
get 



E v vmvmew b 2 VsVr(X js ,X jr ) J P. 

\ \ l<s<r<4 / 

= oU J2 (X jB ,X Jr ) 2 )+o(p~ 2 ). 

\ l<s<r<4 / 



All in all, we obtain that under the event X n , p , 
where 



o(H 2 b 4 h 4 /p 2 ), x = 0, 

o(H 3 b 4 h 3 /np + H 2 b 4 h 2 /p 2 ), x>0, 



Hi = E \(X h ,X ja )\\(X j3 ,X u )\ ( X is,Xjr) 

l<jl<i2<j3<j4<P l<s<r<4 

H 2 = Y \( X ji> X h)\\( X h> X h)\> 

l<jl<j2<33<34<P 

H3 = J2 \(X n ,X j2 )\\(X j3 ,X u )\ E \( X i.> X Jr 

l<ji<j2<jz<j±<P l<s<r<4 

We combine the classical upper bounds, 

\{X h , X j2 ) 1 1 (X j3 , X j4 ) I (X js , X jr ) 2 < (X h , X j2 ) 4 + (X j3 , X j4 ) 4 + ( X ja , Xj 

I ( A^ , Aj 2 ) 1 1 ( X h , X j4 ) I < (X h , X h ) 2 + (Xj 3 , A J4 ) 2 , 
|(A, 1 ,A, 2 )||(A,3,A, 4 )||(A, 3 ,A J? .)| < \(X h ,X h )\* +\(X j3 ,X j4 )\* + \(X js ,X 



4 



>)| 3 
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with ( 13. 6 p and obtain 

E X {H,) = 0(A 2 ), E X {H 2 ) = 0(A), E X (H 3 ) = 0(A 3/2 )- 

Applying Markov's inequality yields 

#1 = Px (A 2 ), H 2 = Px (p 4 n), H 3 = Px {p A r?' 2 ). 

Since b 4 k 2 n = o(l), hp x we get 

h% 8 Hx = 0(hYb 8 n 2 ) = o Px (l), H 2 b 4 h 2 /p 2 = Px (b 4 h 2 p 2 n) = o Px (l). 

If x > 0, we also have to upper bound the term H 3 . Since r 2 p = o{l/y/n) (cf. 
(14.11) ) and since x > 0, we derive that = o(i/n). Then, we get 

E 3 b% z jnp = Px {b 6 p 3 h 3 n^ 2 /nb 2 ) = o Px (l), 

Therefore we obtain A4 = o Px (l). The proposition follows. □ 

5.4 Proof of Proposition 15.21 

We will prove that there exists a family of events Z U}P such that Po(Z niP ) — > 1 and 

v 

log(A(Z)) = lQ g i 1 + ( h / 2 ) ( e4 + edJ - 2 )) °> z G ^ 

We take £„ )P = {(A, F) : < 7}, 1 < j < p, X e Af np } where Af n)P was defined 
in Section IS. 3. 21 It follows from Lemma [5.21 and Section 15.3.21 that P (^ra,p) — > 1- 

Under the events Z np we can replace the quantities (h/2)e d ^ /2 by = 
(h/2)e d J TL± y ' < Tj, cf- Section IB. 3. 1[ Let us consider 

v 

l = J2 Mi + A?) j A 5 = (3/ + ?7 - >0- 

Under the event .4. = *A niP = P|^ =1 (.At + .Aj) defined in Section 15.3.11 we have 
uniformly in 1 < j < p, 

qf + qj = -e~ a V 2 cosh(a,V.) < he~ a V 2 cosh(a,T,-) 
= (1 + e" 2ajTj )/ 2 ~ 1/2, 
as /i — t- 0. Consequently, we have 

L = ^ = A 1 + 0(A 2 ), A 1 = ^A,, A 2 = ^A 2 . 

j=i j=i j=i 
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Thus, we need to show that Ax — y and that A 2 — > in P -probability. It was 
stated in the proof of Lemma [5.31 that E^A 2 = op x (l). Markov's inequality then 
allows to derive that A 2 = op x (l). In order to prove the first relation, we shall 
show that EqAi — > and that Var^Ax — > in P -probability. Observe that 

E*A! = h^WTj - a,) - 1) = -hJ^H-T, + a,). 

3=1 3=1 

By (JUD and (Q we have 

v p 
h H-Tj + a 3 ) x H-Tj) = o(l). 

3=1 3=1 

We have Var^Ai < B + A 2 with B = Ei^kkp 4A and A? = A j ~ E o A j- We 
need to check that, in Px probability, 

E*(B) = <(AA)^0. 

i<j<i< P 

Note that 

E o(AjAi) = Bji - Cji, 

where 

B iX = E* ((g+ + g j)(g+ + gf )) , C fl = h 2 <$>{T 3 - % )<&(T, - a,) . 

We consider independent random variables 771,772 taking values —1 and 1 with 
probabilities 1/2. We write (compare with (15.121) ) 

B jt = h 2 E v [exp {viV2b 2 (X j} X,)) P^)] , C, 7 = /i 2 P°, . 

Here we set 

P° = $(P,-)$(T/) = 1 - $(-fj) - $(-f,) + ^(-fjO^C-f,), T ; = Tt - a h 
We obtain the new decomposition 

Bfl-Cj^tfiUfl + Vji) , (5.21) 

where 

Up = E v [(exp ((^(X,, X,)) - l) P,- , fo)] , = P„ (P w fo) - Jft) ■ 
Let us recall some notations introduced in Section T5.3.41 7^7(77) = rji^Tji, 

II II ll A jll 
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Moreover, Zj and z\ stand for standard Gaussian variables with Cov (zj, z{) = r^rf). 
Then, Pj,i(rj) is written as 

Pj,i(rj) = P*(zj < fj - bm,ji(rj), zi<f t - ftmyfa)) 
= 1 - $(-7} + b mjl (rj)) - $(-T, + b mij ( v )) 

+P*(z J < -fj + brnjiirj), zi < -T, + ftmyfo)) . 

CASE 1: x = 0. The evaluations of the terms in (I5.2ip are similar to the ones 
in Section [5.3.41 We get 

j* ( = i- (p- 2 ), p i ,,(i7) = i- (p- 2 ) J |p^)-pj;,i = (p- 2 ). 

We derive that h 2 Y^i<j<i< P v j,i = °(h 2 )- 

CASE 2: x > 0. We have (compare with (15TT7D and CT| ) 

Qi-fjW-fi) = o{{ph)-% 
p g( z 3 < ~ T i + bm ji(v), *i < ~Ti + bmijiv)) = o((ph)~ 2 ), 

$(-T- + brnjiirj)) = $(— P,) + rn^brji + o(rJ/(p/i)) , 

Taking the expectation over 77, we get 

P„ (P^fo)) - P& = o(r 2 /(M) + o((ph)- 2 ). 

in Px-probability. Therefore 

/> 2 V 3l = 0(Hhp^) + o(l), H= r l- 

i<j<i<p i<j<i<p 

Under X U:P we have r 2 ~ n- 2 (X,-,X,) 2 . Since E x [(Xj,Xi) 2 ] = O(n) for j ^ / 
(Assumption Bl), we get 

R~n 2 {X v X l f = Px {n-Y). 

i<i<'<p 

This leads to h 2 J2i<j<i< p Vji = Px (ph/n) + o(l). Since r 2 p = o(l/y/n) (Eq. 
14.11) and since x > 0, we derive that fc = o(^/ri). Consequently, we have 
ph/n = 0{k/n) = o(l). 

Let us turn to the terms Uj\. They are handled as in Section T5.3.2I We have 

Un = P r ,((r /l r /2 6 2 (X„X i ) + 0(6 4 (X„X,) 2 ) (l + o((p/i)- 1 )) 
= O {b\X j ,X l ) 2 )+0(b 2 \(X j ,X l )\/(ph)) . 

Then, we get 

h2 Yl ^* = 0(WPi) +0(hb 2 H 2 /p) , 

i<i<'<p 
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where 

Arguing as for H, we get 

H x = Px (p 2 n), H 2 = Op x {p 2 n 1 ' 2 ). 

It follows that 

J2 B fl = Px { P Wn) + Px (phb 2 n^) = o Px (l), 
i<j<i<p 

since p 2 h 2 b A n x k 2 b A n — > by (14. ip . The proposition follows. □ 
5.5 Proof of Theorem 14.51 

An in the proof of Theorem 14. 1[ we consider x = limsup x n , p and we take c £ (0, 1) 
such that x c = x/c < <p(/3). We also define b = x c >y\og(p)/n. We first consider the 
case where k\og{p)/n — > 0. 

We use a different prior it than for Theorem 14. 11 Let us note Ai(k,p) the 
collection of subsets of {1, . . . ,p} of size k. We consider a random vector 9 = (9j) 
with coordinates 9j = be,j where ej £ (0, 1). The set of non-zero coefficient of e is 
drawn uniformly in Ai(k,p). This introduces a prior probability it on 9. 

Consider the mixture 

P^dZ) = E w P e VT ^(dZ) = [ P eVI= ^(dZ)n(d9) 
and the likelihood ratio 

0,1 

As in the proof of Theorem 14.11 we shall prove that L n (Z) converges to 1 in P 
probability. This will enforce that ^^ pk [x c ^Jk\og{p)/ \/l — kb 2 } — > 1. Since kb 2 
converges to 0, this will complete the proof. 

The likelihood ratio has the form L n (Z) = YlmeM(k,p) 

\M{k,p)\- l L m {Z) and 
LUZ) = (1 -^exp(-|& + ^M 



x exp 



b 2 

' S 2(l-A;6 2 ) (Xt ' Xj 



(5.22) 



Definition 5.1 Consider 5 £ (0, 1), a positive integer s and anxp matrix A. We 
say that A satisfies a 5-restricted isometry property of order s if for all 9 £ W s , 

0\\ < \\A9\\ < {\ + $)\\6\\ . 
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Let us define the events f2i and £l 2 by 

Sli : " Xj y/n satisfies a 8^ p restricted isometry of order 2k" 



n 2 : "For any 1 < i < p, (Y,Xi/\\Xi\\) < y/2\og(p)(l + <*£>)" , 

where 5n) p = lQ^k \og(p)/n and 5n} P = log~ 1//2 (p). Applying a deviation inequality 
due to Davidson and Szarek (Theorem 2.13 in [6j), we derive that Px(^D = o(l). 
By the Gaussian concentration inequality, we have Po(^E) = Then, we take 

Lemma 5.4 VFe /mve £ [Ll(Z)lI n ] < 1 + o(l). 
Lemma 5.5 VFe /icrae i? [L n (Z)iInc] = o(l). 

Since £? [-^-(■Z)] = 1, we get the desired result by combining these two lemmas. 
Let us turn to the case k\og(p)/n — > oo. We consider b > defined by 

fc^ 2 _ (na ^ fclog(p) 

1 _ jfefi 2 _ ^ ^ „ • 



Lemma 5.6 VFe /icrae 



[#(Z)] 



0(1) 



This lemma implies that for r = \/(2/3 — l)k\og(p)/n — > oo, we have n f^p,k(. r ) ~~ ^ 1- 
□ 

In the proof of the following lemmas, o(l) stands for a positive quantity which 
depends only on (k,p,n) and tends to as (n,p) tend to infinity. 

5.5.1 Proof of Lemma 15.41 

In order to upper bound E [L 2 (Z)]Iq], we first upper bound E Q [L mi (Z)L m2 (Z)Hn] 
for any m l) m 2 e M(k,p). We define W u W 2 , W 3 by , W x = E iemi \ m2 X h 
W 2 = E iem2V ni Xi, and ^ 3 = E i6mi n m2 We note 5 = K n m 2|- 



(Z)L m2 (Z) 



[1 - kb 2 )~ n exp 
b 2 



kb 2 \\Y\\ 2 b(Y, 2W 3 + W x + W 2 ) 



x exp 



kb 2 



\W X + W 3 



kb 2 



\w 2 + w 3 



2(1 - kb 2 ) 

Let us take the expectation of L mi (Z)L m2 (Z) with respect to (Wi, W 2 ). 
E^ Wa [L mi (Z)L m2 (Z)] = (1 - ST^exp 



YfSb 2 2b{Y,W 3 ) b 2 \\W 3 \\ 2 



Sb 2 



1-Sb 2 1 - Sb 2 
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When S = 0, we have E^' W3 [L mi (Z)L m2 (Z)} = I. Let us now consider the case 
S > 0. On the event f2, we have 



W: 



II v.\ 



< y/2Slog(j>)(l + o(l)) 



since X/yJn satisfies a (^-restricted isometry of order 2k. Then, we can upper 
bound the expectation with respect to Y. 



E^[H n L mi (Z)L m2 (Z)] < (l-5V)-"/ 2 exp 

x $ 



b 2 \\W 3 1| 2 
1 + S6 2 



>/251og(p)(l + o(l)) - 2611^11(1 - o(l)) 



Moreover on Q, we have y 1 — 5^ < ||W3||/-\/ri£ < y 1 + Since k\og(p)/n 
goes to 0, we get 

£ [lInW^m 2 (Z)] < exp [x 2 Slog(p)(l + o(l))] $ |Vsiog(p) (V5 - 2x c + o(l)) 



For any x < 0, we have $(x) < e x2//2 . Hence, we get < e ^-Z 2 for any xGl 
It follows that 

E Q [K n L mi (Z)L m2 (Z)} < exp^log(p){x 2 -(l-v / 2x c ) 2 +o(l)}] .(5.23) 
Hence, we get 



E [Z n Ll(Z)] < E, 



P 



S{x 2 c -{l-V2x c ) 2 _+o(l)} 



where S follows a hypergeometric distribution with parameters p, k and k/p. We 
know from Aldous (p. 173) [1] that S has the same distribution as the random 
variable E(U\B P ) where U is binomial random variable of parameters k, k/p and 
B p some suitable er-algebra. By a convexity argument, we then obtain 



E [K a Ll(Z)} < 



1 + h. ^-(l-V^x c ) 2 _+o(l) _ 



< exp 

< exp 



k 
p 

k 2 ^~{l-y/2x c )l+o(l) 



P 



-p 



V 



l-2/3+x2-(l- v / 2x c )l+o(l) 



Since x c < <p(/3), one can check that 1 — 2(3 + x 2 — (1 — v^c) 2 - is negative and we 
conclude that E [S. n Ll(Z)] < 1 + o(l). □ 
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5.5.2 Proof of Lemma 15.51 

By symmetry, it is sufficient to prove that E (L m (Z)llQc) = o(l). Let us decom- 
pose E (L m (Z)H n c) = E (L m (Z)H n c) + E (L m (Z)H ntun2 ). Since E*(L m (Z)) = 1, 
Px almost surely, we have E (L m (Z)HQc) = P X (Q,^) = o(l). Let us turn to 
E {L m (Z)K n c vn2 ). For any 1 < i < p, we define the event fi« by (Y, > 

v^i(p)(i + *S)- 

p 

i=l 

The value of these expectations depends on i through the property "i £ m" or 
"i ^ m". Let us assume for instance that l6m and 2 ^ m. Then, we get 

£ (L m (Z)n n c un2 ) < fcEo [L m (Z)H n2 lI n(1 )] + p£ [L ro (Z)I„ a I n(a) ] . (5.24) 

First, we upper bound E [L m (Z)'S.Q 2 'S. Q (2)}. Taking the expectation of L m (Z) with 
respect to (Xi) iem leads to EQ ,X2 [L m (Z)] = 1. Hence, we get 

E [L m (Z)K Q2 lL a(2) } < P ({lW) < p -i e -V^W = oip' 1 ) . (5.25) 

Let turn to E [L m (Z)Hf l2 ll n (i)}. We first take the expectation of L m (Z) condition- 
ally to X\ and Y: 

E^[L m (Z)] = (l-b 2 )- n/2 ew 
Then, we take the expectation with respect to Y 



E^[L m (Z)l aa )\ <l-$ 
Moreover, on Q2 we have \\Xi\\ < \/n(l + o(l)) 

-log(p)( v / 2-x c -o(l)) 2 /2 
for (n,p) large enough, since x c < (f(/3) < \[2. 

kE^[L m (Z)H nWu n 2 ] < p-^-^ 2 /^+°V = o(l) , (5.26) 

since x c < V2(l-y/T=J) < <p(P). Combining (ETM . fl5^5l) . and (KM completes 
the proof. □ 



b 2 \\Y\\ 2 (Y.X^b 
2(1 -b 2 ) 2(1 -b 2 ) + 1-b 2 



B A '[L„(Z)n 



Q( 1 )uQ 2 . 



< $ 



>/log(p)(x c - v^ + o(l)) <Cexp 
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5.5.3 Proof of Lemma 15.61 



Arguing as in the proof of Lemma I5.4[ we get 

E^[L mi (Z)L m2 (Z)} = (l-S 2 6 4 )-"/ 2 exp 

Taking the expectation with respect to W3 leads to 

E [L mi (Z)L m2 (Z)} = (1 - Sb 2 y n/2 < exp 



b 2 \\W 3 \\ 2 
1 + Sb 2 



nSb 2 
2(1 -kb 2 )_ 



As in the proof of Lemma EH we upper bound the term E [L 2 (Z)] by Jensen's 
inequality. 



Eo[Ll(Z)} < 



k f f nb 2 

1 + p \ exp [w^wj 



< 



cxp 



e f nb 2 

7 exp V2(i-^ 2 ) 



< exp [/"^exp {((3 - 1/2) log(p)}] = 1 + o(l) , 
since 6 satisfies kb 2 /(l — kb 2 ) = (2/3 — l)klog(p)/n. □ 

6 Proofs of the upper bounds 

6.1 Tests based on the statistic to 

Recall that 

n 

to = (2n)- 1 / 2 ^(F 4 2 -l). 
i=i 

Under H , the statistics Yi — £3 ~ A/"(0, 1) are i.i.d. This implies 
Eo{to) = 1, Var (to) = 1- By the Central Limit Theorem, t ~~ * £ ~ A/"(0, 1) as 
n —7- 00 in P -P r °bability This yields Theorem 14.21 (i). 

Let us consider the type II errors. We need to show that, if nr A — > 00, then 
su P0ee (r) Po(to — u a) 0. We will prove that, uniformly over 8 G 6 p (r), 

E e t ->■ 00, Var e t = o((E e t ) 2 )- 
Indeed, if (16.1 j) is true, we derive that for n,p large enough, 



(6.1) 



Ptf(*0 < «a) = - <0 > ^0 - ^) < n(|^0 ~ *o| > E e t - U c 

Var e (t ) 

o(l) , 



< 



(Egt - U a ) 2 

by Chebychev's inequality. In order to check (16. ip . we use the identities 
E e t = E x (E?t ), Vai e t = Var x (E*t ) + E x (Varf t ) . 



(6.2) 
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Under , 9 G 0fc(r), we have Y ~ N n {v, I n ), where 

v P 
v = v(6,X) = Y, 9 i X ii ll^ll 2 = E^II^II 2 + 2 E OM x » x i)- 

3=1 j=l Kj<l<p 

It follows that 

Ef{t Q ) = {2n)~ 1 ' 2 \\vf, Var^(t ) = 1 + 2rT 1 |M| 2 . 

Since ^x(||^|| 2 ) = n, E x ((Xj,Xi)) = 0, j ^ Z, we get the first convergence in 
(EU): 

Egt = {2n)- 1 ' 2 E x {\\vf) = = (n/2) l l 2 \\6\\ 2 > {n/2) l ' 2 r 2 -+ oo. 

3=1 

Let us turn to the variance term 

^x(Varfto) = l + 2n" 1 E x (||t;|| 2 ) = l + 2||^|| 2 = o(^ ), 
Var x (£*(t )) = (2n)- 1 Var x (|| W || 2 ). 

By A2, the random variables Xjj are independent in % — 1, ...,n, j = 1, 

Consequently, the random variables (Jf^JQj with 7^ {32, h} are uncorre- 

lated. Moreover, HXjJI 2 and (Xj,X{) are uncorrelated as long as (J, I) 7^ ( Ji , Ji ) - 
We have 

Var x ||Xj 2 = Var x (X 2 K £ X (X„X) 2 = n, j ^ I, 
where Var x (X 2 ) < #x(Xg) < 00 by Bl. Then, we get 



p 

D 2 



n~Waix\\v\\ 2 = n-^^VarxUX^I 2 + 4n _1 d fti E x(Xj,X t 

3=1 1<3<«<P 

< sup^xpq*)] £ £ 4 + 4||£|| 4 < (0(1) + 4)||£|| 4 

* 3=1 
= (n\\9\\ 4 ) = o ((E e t ) 2 ) , as n||#|| 4 > nr A -> 00. 

Therefore we get the second relation (16. ip . 

Note that if nr 4 — » 00, then in the inequality (16. 2p . we can replace u a by 
a sequence T np — >■ 00 such that limsupTnpr -2 ?^" 1 / 2 < 1, for instance by T pn = 
n 1 / 2 r 2 /2. Then, the corresponding test satisfies 7(V ;0 , @p( r )) - ^ 0. Theorem 14.21 
follows. □ 

6.2 Tests based on the statistic t\ 

First observe that under H , the statistic t\ is a degenerate [/-statistic of the second 
order, i.e., for Z s = (X^ S \Y S ), s = 1,2,3 one has Ez 1 K(Z±, Z2) = 0, which yields 



30 



E ti = 0. By Assumption Al, 



p v 



E t\ = E (K 2 (Z 1 ,Z 2 ))=p- 1 E (Y l 2 Y 2 2 )J2J2 Ex ^ X ^ XuX ^ 

3=1 1=1 

Set 

G(Z U Z 2 ) = E z , (K(Z U Z 3 )K(Z 2 , Z 3 )) , G 2 = E (G 2 (Z 1 , Z 2 )), G 4 = E^K\Z U Z 2 )), 

where Ez :j denotes the expectation over Z 3 under P . In order to establish the 
asymptotic normality of ti we only need to check the two following conditions, see 
[H] Lemma 3.4, 

G 2 = o(l), G 4 = o(n 2 ). (6.3) 
We have by Assumption Al, 



v v 

G{Z 1 ,Z 2 ) = p- 1 E z jY 1 Y 2 YiJ2J2 X ^ X ^ X ^ 

\ j=i i=i 

p p 

= p-^Y^Y,^ 21 ^^^) 

j=l 1=1 

p 

= p~ 1 Y 1 Y 2 Y,X lj X 2j =p- 1 / 2 K(Z u Z 2 ). 

3=1 

Since E (K 2 (Zi, Z 2 )) = 1, we get the first convergence in (16.31) . Next by A2, 

p p p p 

E (K\Z 1 ,Z 2 )) = p- 2 J E (F 1 4 r 2 4 )^^^^E x (X 1 ,X 2j X 1 ^ 2 ,X lr X 2r X ls X 2s ) 

j=l 1=1 r =l s=l 
P P P P 

j=l 1=1 r=l s=l 

since E Q {Y?Y 2 A ) = E 2 {Y?) = 9, where we set 



(E x (Xf), j = l = r = s, 
Hjirs = Ex{XijXuXi r Xi s ) = ( 1, j = I ^ r = s or j = r ^ I = s or j = s ^ r = I, 

[0, otherwise. 

As a consequence, we get 

E (K 4 (Z ll Z 2 ))<9p- 1 bl + 27 , 



31 



where 6 4 = s\xp i E(Xf 1 ). By Bl, the second convergence in (16. 3p holds true. Thus, 



Theorem 14.31 (i) follows. 

Let us now evaluate the type II errors under Pg. Recall that by (11. ip . 

v 

Y i = Z i + v i , Vi = J2 i X v> &~JV(0,1) Ud. 

3=1 

Observe that EgYiX^ = 9j and set 

p 

K e (Z 1 , Z 2 ) = v~ X ' 2 J2^ X ^ - d i)^ X V - e i)- 

3=1 

Consider the representation 

K(Z 1 , Z 2 ) = K e (Z 1 , Z 2 ) + 5{Z X ) + 5(Z 2 ) + h{6) 

where 

m) = p~ 1/2 YSXi^ - e 3 )e 3 , h{e) = p~ 1/2 £ b) . 

3=1 3=1 

Observe that the kernel Kg(Zi, Z 2 ) is symmetric and degenerate under Pg, i.e., 

E^K e (Z 1 , Z 2 ) = E z ^Kg{Z 1: Z 2 ) = 0. 

The terms Kg(Z 1 , Z 2 ), S(Zi), and 5{Z 2 ) are centered and uncorrelated under Pg. 
As a consequence, we derive that 

E e {K{Z u Z 2 )) = p^ 2 \\9f, (6.4) 
Vbx 6 (K(Z u Z 2 )) = Varg(Kg(Z 1 ,Z 2 ))+VaTe(S(Z 1 ))+VaT e (6(Z 2 )) . (6.5) 

Let us compute the variances. Let 6{j be the Kronecker function. Using the repre- 
sentation 

p I p 

Kg(Z 1 , Z 2 ) = p" 1 / 2 tl X l3 + d r{ X lrX l0 ~ 6 rj ] 

j=l \ r=l 



X 



£ 2 X 2 j + 9 s (X 2s X 2 j — 5 S 



s=l 



we derive that 



p p p 

E^(K e (Z 1 ,Z 2 )) = p-w^^£e r e 8 {x lT x li -5 rj )(x 2a x 2j -8 Bj) 

j=l r=l s=l 
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Denoting H rs j = (Xi r X V j — 5 r j)(X 2s X 2 j — 5 S j) observe that 

p p p p p p 
Vax x E?(K e (Zi, Z 2 )) = P~ l Y,Y,Y,Y,Y,Y, 0r0 s u v E x (H rsj H uvl ). 

j=l r=l s=l 1=1 u=l v=l 

Note that 

Ex\H rs jH uv i) D ru jiD SV ji y 
where (we omit the first index i — 1, 2 in Xij) 

D ru ji = E x {{X r Xj — 5 r j)(X u X[ — 5 U [)) , 
D SV ji = E x ((X s Xj — S s j)(X v Xi — 5 v i)) . 

Observe that 

{1, r = I ^ u = j or r = u j = I, 

6 4 - 1, r = u = j = I, 
0, otherwise. 

We obtain 

p p p p p p 

Vw x E?(K e (Zi, z 2 )) = p- 1 £ £ £ £ e r e s e u e v D rujl D svjl 



j=l r=l s=l 1=1 u=l v=l 

^^ 2^-1 ^ 1 ^ 

» ^-^ p ^-^ p 

p p 



= o(£0}] + o[(£^) t \ = o(\\9\\% 

3=1 3=1 

We now compute E x \Vai e (Kg(Zi, Z 2 ))]. 

V^f{K e {Z 1 ,Z 2 )) = p- 1 X Xj X 2j X n X 2l 

i<j,i<P 
p p p p 

+ P^J2J2J2Y1 XijXM(X 2r X 2j - 8 jr )(X 2s X 2l - 5 sl ) 

j=l 1=1 r=l s=l 

p p p p 

+ P~ l ^ ^ ^ ^ X 2 jX 2 i9 r 9 s (Xi r Xij — 5j r )(Xi s Xii — 5 S [) 

j=l 1=1 r =l s=l 

Let us take the expectation with respect to X. By Assumption A2, we have 

p p p 

E x [Vw:f(K e (Z 1 ,Z 2 ))} = l + 2^ 1 ^^^^.^ J E x [(X 2r X 2j -5 jr )(^2 S A 2j -^ 

j=l r=l s=l 

V 

< 1 + 2^64^ = 1 + 0(||^|| 2 ) = 0(1 + 

r=l 
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Since 

VMMZi, Z 2 )) = ExVarfiKeiZ!, Z 2 )) + N^ x Ef{K 6 {Z x , Z 2 )), 

we get 

VareiKeiZ!, Z 2 )) = 0(1 + ||#|| 4 ). (6.6) 

Similarly for i = 1,2, we compute the variance of 5{Zj). 

p / p 

s(Zi) = P - 1/2 % tiXij + Yl 9 i( x v x u - h) 

3=1 \ 1=1 

and we have (we omit the index % = 1,2) 

Ef{5{Z)) = p-^Y^BMXsXi-S^ 

3=1 1=1 
V P 

Vmf{6{Z)) = ■p- x Y^O j O l XjX l , E x V a if(5(Z))=p- 1 

3=1 1=1 
P P P V 

Vw x Ef{5{Z)) = p-^^Y^BjOAOsDjri. , 

3=1 1=1 r=l s=l 

where Dj ris was previously defined and upper bounded. This yields 

Y^ x Ef{5{Z))=p~ 1 ^4-1)^^ + 211^11^ =0(||^|| 4 /P). (6.7) 

Combining (16.41) . (16.51) . (16.61) . and (16.71) we obtain, for r 2 np~ l l 2 — > oo and 
p = o(n 2 ), 

E e {h) = 4NE e {K{Z x , Z 2 )) = VNh(6) ~ n(2p)-^ 2 \\e\\ 2 > -^=r 2 -)• oo, 



Var e (ti) = Y^e{Kg{Z l ,Z 2 )) + -V^{Z l )) = 0{l + \\9\\ A )+0{n\\e\\ A /p) 



o 



{(E e (ti)) 2 ) 



Applying Chebyshev's inequality as in the proof of Theorem 14.21 allows to conclude. 
□ 

6.3 Higher Criticism Tests 
6.3.1 Type I errors 

The variables Xi, ...,X P ,Y are independent under P and (Xj, a)/\\a\\ ~ Af(0,l) 
for any a G IR P , a ^ under A3. Thus we have 

P (y 1 <t 1 ,...,y p <t p ) = E Y (P^((X 1 ,Y)/\\Y\\<t 1 ,...,(X p ,Y)/\\Y\\<t p ) 

= £ y = $(fi)....$(*„) = P {yi < h)...P G {y p < t p ) 
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It follows that yj = (Xj, Y)/\\Y\\ ~ jV(0, 1) and yi, . . . , y p are i.i.d. under P . As 
a consequence, the random variables qi are independent uniformly distributed on 
(0, 1) under P . We denote by F p (t) the empirical distribution of (qi)i<i< p : 



P i=l 

Then, the normalized uniform empirical process is defined by 

W p (t) = Jp- 



F p (t) - 1 



Arguing as in Donoho and Jin [Hj, we observe that tnc = su Pt<i/2 W p {b). It is 
stated in (26], Chapter 16 that 

sup <t<i/2 W p (t) 

> P 1, p -> oo . 

V^iog logp 

This proves the result. □ 
6.3.2 Type II errors 



We define H np = (1 + a) a/2 log log p. Consider some (3 G (1/2, 1) and assume that 
k\og(p)/n — > 0. It is sufficient to prove that for any 5o > arbitrarily small the 
radius 

r np = ((p(p) + 5oWk\og(p)/n (6.8) 

satisfies 

(3(i; HC \e k (r np ))^0 . (6.9) 

For any 9 G we set ||#||oo = supj^l. In order to prove the convergence 
( 16.91) . we consider a partition of 0fc(r np ): 

6?>(r„) t e,(r,)n{« 6 e„||»||»> i ^} 



ef (r np ) 4 e fc (r np ) n [e>i 1) (r np )] c n [ef\r np )] c . 

The sets 0^ (r np ) and ©j^ (r np ) contain the parameters 9 whose l 2 or norms are 
large, while the set ©£ (r np ) contains the remaining parameters. 

Proposition 6.1 Consider the set of parameters 0^ defined by 
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Let us introduce the statistic t max and the corresponding test -0 max defined by 

W = (pq W r 1/2 - (M(i)) 1/2 < t UC , iT" = H tm>Hnp . 
We have /3(^ max , 6^) 0. 
It follows that /3(> HC , 6^ 4) ) 0. Observe that 

(1) fr IcI^GG, ll^ll 2 > 4fclog(p)/n 

Since fc||0||^o > \\9\\ 2 and since k\og(p)/n converges to 0, it follows that 
©i ( r np) C 6^ for n large enough. Thus, we get (3(tp HC , (r„ p )) — > 0. 

Let us turn to Q k (r np ). For any 9 E O k (r np ), we have 

||0||^, > 41og(p)/n 



1 + ||6>|| 2 ~ 1 + 4Hog(»/n ' 

This quantity is larger than 31og(p)/n for n large enough. We get 

/3(^,ei 2) (r np ))^0. 

Proposition 6.2 Lei us se£ T p = ^/log(p) and u > sitc/j t/iat 

• Jf. ^(1/2,3/4], 
\V2, (9 6(3/4,1). 

VKe consider the statistic L(u) and the corresponding test ip L defined by 

i(M) - g V^FSg) ' * ~ L "' }> - H -' ■ 

Then, /3(ip L , Q k 3 \r np )) — > 0. Moreover, we have L(u) < t HC , forp large enough. 

It follows from Proposition O that /3(ip HC , Q k \r np )) ->■ converges to 0, which 
completes the proof. □ 

6.3.3 Proof of Proposition 16.11 

It follows directly from the definition ( 14. 7p that t max < t HC - Consider the test 
ip' max defined by 

^' maX = IT „ > r-— (6.11) 

If ^' max = 1, it follows that q {1) < 2$(- ^2.5 log(p)) < 2p" 5/4 . Hence, we have 
^max > P 1 ^ 8 /"\/2 — V^p" 1 / 8 . For p large enough, this implies that ?/; max = 1. 
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Consequently, we only have to prove that /3(?// max ,ej; 4) ) -> 0. 

Consider 9 G ©L^- By symmetry, we may assume that ||#||oo — We use the 
following decomposition 

\\Y\\ yi = 9 1 \\X 1 \\ 2 + (Y-9 1 X ll X 1 ) . 

The random variables ||y|| 2 /(l + ||#|| 2 ) and ||Xi|| 2 have a \ 2 distribution with n 
degrees of freedom. Since Y — 9±X\ is independent of Xi, the random variable 
(Y — 8\Xi, Xi/\\Xi\\) is normal with mean and variance 

With probability larger than 1 — 0(n _1 V log we obtain 

\\Yf/n < (l + \\9\\ 2 )[l + o(n- l l A )} 
(1 - o(n" 1/4 )) < \\Xyf/n < (1 + o(n' 1/4 )) 

\(Y-9 1 X 1 ,X 1 )\/\\X 1 \\ < (l + ^^^logQogb)) • 

Thus, we get 

M > (i ^2)1/2 1 1 - °(^ 1/4) ] " o(>A>giog(p)) , 

with probability larger than 1 — V log -1 ^)). Since G ©^ , we have 

n|6'i| 2 /(l + ||0|| 2 ) > 31og(j>) and the test ip' max rejects with probability going to 
one. It follows that (3(tp' max , 6^ 4) ) -> 0. □ 

6.3.4 Proof of Proposition [6721 

Connection between tuc an d L(u). Set s u = ^f =1 H^.^uTp- Observe that 
q(s u ) < P(\N(0, 1)1 > uT p ) < 1/2 for p large enough. If follows that 

r / \ y/v[sjv~ 2$(-«T p )] y/p[sjp - ?(*„)] . , 

-M^J = = S S 1>HC ■ 

^2<$>{-uT p ) 

Power of Under P#, ||Y'|| 2 /(1 + ||#|| 2 ) has a x 2 distribution with n degrees 
of freedom. For any 9 G (r np ), we have ||#|| 2 < 4Mog(p)/n = o(l). As a 
consequence, we have |||F|| 2 — n\ < 4 A; log (p) +4-y/n log(n) = o(n) with probability 
larger than 1 — 0(1/ n) uniformly over all 9 G 0^ (r np ). Consider the event Z np> i = 
{|||F|| 2 — n\ < H n }, where H n = 4k\og(p) + 4^/nlog(n) = o(n). It is sufficient to 
prove that 

sup P e (Z n p,i n {L(u) < P„ p }) 0. (6.12) 

eee^ 3) (r„ p ) 

Consider G 0^ (r np ). We can assume that 9k+i = ■■■ — 9 P — 0. Then 
F = 5~j* =1 ^--Xj + £ does not depend on X fc+1 , ...,X P . Arguing as for the type I 
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error, we derive that yu+i, ■■■,y p are independent standard Gaussian variables and 
do not depend on (yi, ...,yk)- We can write L(u) = Li(u) + L 2 (u), where 

T ( , ZU (K {M>uTp} - 2$(-uT p )) 
L x {u) = 



y/2p$(-uT p 

L 2 (u) -- 



{^ { \ yj \>uT p} - m-nT p )) 



^2p<5>(-uT p ) 
We find 

£,(L 2 («)) = 0, Var e (L 2 ( M )) = 2^(-uT p ) " ' 

which yields, 

P e (|L 2 ( M )|># np )^0. (6.13) 

In order to study the term Li(u), we will find a statistic Li(u) such that 
Pg[Li(u) < Li(u)] = 1 + o(l) uniformly over 0}^ (r np ). For such a Li(u), we 
will have 

P e [L(u) < # np ] < P e [Li(u) < 2# np ] + o(l) < P e [Li(u) < 2H np ] + o(l). (6.14) 
Construction of L\(u). Observe that under Pg, 

yj = felKII + ^ + A,)/||F|| , 

k 

A j = J2 9 i( X i> X i) + {\\ X j\\ 2 - n ) 9 j> J = h-,k, 

where 

%- = (x„o/IKI|. 

We only need to consider Z E Z np ^ = {||£|| 2 — n\ < n 2 ^ 3 } since Pe[Z np ^) — > 1. Set 
Z nP;3 = Z nPi i n Z np ^. Thus, for 5 = 5 np -> one has 



{| % | > «T p } n Z„ p , 3 D {In-^-Hell + rT 1/2 A,- + n^%)\ > uT p (l + 5)} n Z np , 3 

D {sgnidj)^! -5)> uT p (l + 5)- n 1/2 |%| + n 3 



np,3 j 

(6.15) 



where Si = n 1 / 2 A, 



Lemma 6.1 For any T > jomj to infinity and such that T = o{y/n), we have 

\og(P x (\S J \>T\\e\\))< -^(1 + 0(1)) , 
uniformly over E Q^\f n p)- 
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Taking T = ^JA log(p), we obtain 

p x (\s j \>T\\9\\) = o( P - 1 ). 

We recall that ||0|| 2 < Ak\og(p)/n = o(l) since 9 G Qf\r np ). Hence, we get 



P 



x 



max \Sj\ > o(\/log(p)) 

l<j<k 



o(l), uniformly over 



Combining this bound with (I6.15p . we obtain that there exists an event Z npA of 
probability tending to one and a sequence 5 = 5 np — > such that 

{\ yj \ > uT p } n Z npA D {sgnie^yj > uT p (l + 5) - (1 - 5)n 1 / 2 |%|} n Z npA . (6.16) 

Observe that the random variables yj are independent standard normal. 

Setting u — u(l + 8), pj — (1 — <5)ri 1//2 1 ^ | we define 



(e U n 



^2p<&(-uT p 



By (16.161) . Li(n) satisfies Pg[Li(«) < = 1 — o(l) uniformly over O*. (r : 

In view of ( 16.14(1 . in order to complete the proof it suffices to show that 

Peihiu) < 2H np ] = o(l) uniformly over ef\r np ). (6.17) 

Control of Pq[L\(u) < 2i? np ]. In order to evaluate this probability, recall that 
y~j ~ A/"(0, 1) i.i.d. under P 9 . Thus, 

(E-=i$(-ST p + p J )-2$(-uT p 



np J 



Var^L^w)) < 



^2p<$>(-uT p ) 



2p$(-uT p ) 
By Chebyshev's inequality, we get 

P e (L 1 (u)<2H np ) = P e (Eg(L 1 (u))-L 1 (u)>Eg(L 1 (u))-2H np ) 

Var e (Li(n)) 
" {E e {L x {u))-2H np f 
Lemma 6.2 There exists t] > stzc/i £/ia£, for n,p large enough, 

§{-ut p + Pi) 

inf ^ J ~) P =^- ~ inf S„(Li(m)) > p". (6.18) 

»6©fc 3) (»vw) <s/2p$(-uT p ) eee[ 3) (r np ) 

In the sequel, we denote by A p a log-sequence, i.e., a sequence such that A p = 
(log(p)) Cp , \cp\ = 0(1) as p 00. Since u G [0, a/2], we have p$(— uT p ) > A p . 
Combining this bound with Lemma 16.21 yields 



Va,r e (Li(u)) = O [ApEeiL^u)) 
Since if np = o(p v ), this implies (" 16 . 1 7(1 and then (I6.12p . □ 
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6.3.5 Proof of Lemma 16.11 

Let us bound the deviations of Sj by computing the exponential moments of A 
For any h such that /i 2 ||#|| 2 < 1/4, we have 

E x (exp{hAj)) = E Xj Ex j {eMhAj)) 



E Xj ^exp^dl^f-^^exp (^^9^, 
E Xj ( exp [W i (||X i || a - n) + f ||X#5> 2 



exp 



. _ | i og _ 2Mj -h 2 J2 j 



as 2/i#j + /i 2 JZj^i fy 2 < 1- Taking /i such that h 2 k\og(p) = o{n) enforces /* 2 ||#|| 2 
o(l). Using the Taylor expansion of the logarithm 

-hx - \ log(l - 2hx - h 2 y 2 ) = h\2x 2 + y 2 ){l + o(l)), h\2x 2 + y 2 ) = o(l), 
we get 



ExiexpihA,)) = E x (exp(^hS ] ))=expl^h 2 (2e 2 + J2 e i^ 1 + ^ 

< exp [n/i 2 ||0|| 2 (l + o(l))] (6.19) 

as /i 2 !!^ 2 = o(l). Take some T > 0. Applying a standard technique based on 
Markov's inequality yields 



Px{Si > T\\e\\) < E 



x 



p x {-s, >T\\e\\) < E x 

We get from fl6U9j) that 





( TS, T 2 \ 


exp 


\?n 2 ) 




( TS 3 T 2 \ 


exp 


V 2||0|| + 2 ) 



logCPxd^-l >T\\6\\)) < --T 2 (l + o(l)) if T 2 = o(n) and T — )• oo. 



□ 



6.3.6 Proof of Lemma 16.21 

Recall that we consider r np = (ip(/3) + 5 )a/^ l°g(p)/ n with arbitrarily small 5 > 
(see (16.81) ). Recalling that T p = ^/log(p), we apply the results of Section [731 for 
<5 = 5„ p > 0, 5 np = o(l), and 

T = uT p , u=(l + 5)u, v — (1 - 8)(<p(0) + 5 Q ) < u, t = vT p , R = 2T P , 
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since for tj = pj one has 

^ = (l-SrnJ2e]>(l-6)V np = ktl 

3=1 3=1 

By the choice of u, and v, the relations (I7.4p hold true for p large enough (see 
Remark 17.11) . Applying Lemmas 17.41 and 17. 5^ we get 



inf $ (~ ST P + Pj) = k§{-uT p + to). 



9ee k (r np ) * =1 



We recall that A p denotes any log-sequence. Since $(— tT p ) = A p p * 2 / 2 for t > 0, 
we have 

~ fc($(-uT p + t )-2$(-uT p )) M>(-uT p + t ) 

mf E e {Li(u)) 



eee h (r np ) ^2p<$>(-uT p ) ^2p<5>(-uT p ) 

k$(-(u-v)T p ) = A 1/2 _ /3 _ (s _,, ) 2 +/2+ , t 2 /4 

In order to obtain (I6.18p . we have to check that there exists rj > such that, for 
n,p large enough, 

2 H 2 4 ~ ' 

Let (3 G (1/2,3/4]. Recalling that </? 2 (/3) = 2/3 - 1 > and (ISTTUj) we see, that 
for 5 = 5 np = o(l) and 5 G (0, <p(l3)), one can find 77 = r^(/3, 5 ) > such that 



^ 2 (/3) (<p(j3)-5 ] 



2 



G = _n^_ ^ +y 2 Gg) + o(1) 

= 1 + 0(1) > 77 + 0(1). 

Let us now consider (3 G (3/4,1]. Recalling that <p(0) = y/2(l - VT^fi) 
and (16.101) . we see that for 5 = 5 np = 0(1) and 5 G (0, y/2 — 2(3), one can find 
r\ = r)((3, S ) > such that 

G = l_,_ (V2-V2(l-^-M 2 + l +0(1) 



1-/3- +o(l) 
y/2 - 2/3 5 - I + o(l) > 77 + 0(1). 



The relation ( 16 . 1 8[) follows. □ 
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6.4 Proof of Proposition 14.61 

Under H , the distributions of the variables (yi)i=i t ... tP do not depend on a 2 . As a 
consequence, E 0j(T (ij; HC ) = E ^. This last quantity has been shown to converge to 
in Theorem 14.41 Hence, we get a un (ip HC ) = o(l). 

Let us turn to the type II error probability. We consider the model Yi = 
Y?j=iQjXij + & where Var(^) = a 2 . Dividing this equation by a, we obtain the 
model: 

p 

3=1 

where Var(^) = 1. The statistic tjjc is exactly the same for the data Z = (Y,X) 
and Z' = (Y',X). Consequently, we obtain E etTytT (l - ip HC ) = E e>1 (l - ^p HC ). It 
remains to use the bound on — ijj HC ) from Theorem 14.41 □ 

7 Appendix: Technical results 

7.1 Thresholds 

Take the thresholds T = Tj satisfying 

Oj log^ 1 ) 
3 o 



2 o, 



Define aj = Xj^/log(p), Tj = Tj/ y4og(p), and h = p /3 . Then, we have 

Tj = Xj/2 + fi/Xj. 



If for some 5o > 0, Xj + 5q < <f2{P) = \/2(l — VI ~~ /3) < ¥>(/^) > then there exists 
Si > such that Tj > \[2 + t^. For such a Xj, we derive that 

pTJ^-Tj) = o(l), Vr>0. (7.1) 

In particular, if Xj = o(l), then Tj — > oo and ( 17. ip holds. 

For any 5 > 0, we have 

$(-Tj) x /i$(-Tj + «j) for Tj > Xj + 5 . (7.2) 
This holds if Xj < tp 2 (0) < y/2. 

7.2 Norms \\Xj\\ and scalar products (Xj,Xi) 
Clearly, 

S(||Xj|| 2 ) = n, E{Xs,Xi) = , Var^X,) = n. 

By Assumption Bl, there exists D > such that supj^ z Var(Xj, X;) < n.D and 
supj Var(||Xj|| 2 ) < nD. 
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Lemma 7.1 Let Uj be a random variable distributed as Xij. 

(1) Assume that there exists h > such that sup 1< <Kp E(e hU]Ul ) < oo for any 
\h\ < h . Then, for any sequence t = t n such that t = o(y/n) and ty/n — > oo ; 



1 2 „i ^ j. rz:\ ^ „ „r j-2 

r \\\\*i\ 

and 



P{\\\Xj\\ 2 -n\ > ty/n) < exp[-t 2 /(2 J D)(l + o(l))], 



P(\(X j ,X l )\\>ty/n) < exp[-t 2 /(2 J D)(l + o(l))], 

(2) Assume that E{\X\ m ) < oo, for some m > 2. Then there exists C m < oo 
such that 

P(|||XJ 2 -n\ > tVn) < C m t~ m '\ P(\(X j} Xi)\ > t^n) < C m t~ m . 

Proof follows from the standard arguments based on the moment inequalities and 
exponential inequalities. If EZ = 0, Var(Z) = 1, E(e h ° z ) < oo, then \og(Ee hZ ) = 
h 2 /2(l + o(l)) as h — > 0. Hence, we take h = t/y/n = o(l) for the study of the 
exponential moments of S n = Y17=i ^i- 

Corollary 7.1 

(1) Let log(p) = o(n) and the assumptions Lemma \7J\ (1) hold true. Then, for any 
B > 2, one has 

Px(max |||XJ| 2 -n| > y/BDn log(p) ) = o(l), 

i<i<p 



P x { max \(X j ,X l )\> ^/2BDn\og(p)) = o(l). 

i<j<i<p 



(2) Let p = o(n m / 4 ) and the assumptions Lemma \7. 1\ (2) hold true. Then, for any 
sequence v n going to infinity, one has 

Px(max |||X-|| 2 —n\> s/np 2lm v n ) = o(l), 
i<i<p 

P x { max \{X j7 X{)\ > V^p 2/m v n ) = o{\). 
i<j<i<p 

(3) Under assumptions (1) or (2) uniformly in 1 < j < I < p in Px -probability, 
one has aj ~ a = b^/n, Xj ~ x, i.e., for any 5 > 0, 

P x (max \(aj/by/n) - 1| > 5) ->• 0, Px(max \{xj/x) - 1| > 8) -»■ 0. 

i<j<p i<j<p 

7.3 Expansion of 

Let $(t) be the standard Gaussian cdf and 4>(t) be the standard Gaussian pdf 

Lemma 7.2 Let 5 -> 0, tS = 0(1). T/jen 

$(t + 5) = $(t) + <ty(t) + O (<5 2 (|i| + l)0(t)) . 

Proof follows from the Taylor expansion and the properties of 4>(t). □ 

Observe that for any b G 1R there exists C = C(b) > such that (\t\ + 1)0(— £) < 
C(6)$(-t) as i < b. It follows from Lemma O that as 5 -> 0, tS = 0(1), i < B 
for some G H, then 

$(_t + 5) = $H)(1 + 0(5 2 )) + 50(t). 
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7.4 Tails of correlated vectors 

Lemma 7.3 Let (X, Y) be the Gaussian random two-dimensional vector, 

EX = EY = 0, Var(X) = Var(F) = 1, Cov(X, Y) = r. 

Let t\ x t 2 — > oo, rt± = o(l). Then 

P{X >h,Y> t 2 ) = $(-i 1 )$(-t 2 ) (1 + 0(r 2 )) + r^t^h). 

Proof. Observe that the conditional distribution £(Y\X = x) is Gaussian 
Af(m(x), o~ 2 (x)) with mix) = rx, a 2 (x) = 1 — r 2 . Therefore 

P(X > t u Y > t 2 ) = jf P(Y >t 2 \X = x)d<$>(x) = $ + T ) 

Setting ft, = |r| _1 , observe that 

$ - = (^ $ (-^i) $ (-^)) • 



It is sufficient to study the integral over the interval A = [ti,/i]. For x G A, we 
have 

= = -t 2 + S(x), S(x) =rx + 0(r% + \r 3 x\)) = 0(1). 



vT 



Applying Lemma 17. 2\ we have 

= $(-t 2 ) ($(-*!) (l + 0(r 2 )) + r0(t 2 ) / xd$(a:) 
= $(_ tl )$(_t 2 ) (i + o(r 2 )) + r0(*i)0(t 2 ), 
since J A xc?$(x) = - = 0(*i) + o(r 2 $(-ti))- □ 

7.5 A minimization problem 

Let /(t) be a function defined on the interval t G [0, i?]. Consider the minimization 
problem 

k k 
F k (t ) = ini^fitj) subject to ^t] > fctjj, t s G [0,R). (7.3) 

j=i i=i 

Lemma 7.4 ^ss^me t/iai t/iere exists A > swc/i t/jat 

inf (/(t) - At 2 ) = /(t ) - At 2 . 

T/ien F fc (i ) = kf(t ). 
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Proof. We have, for any (t 1; ...,t k ) such that tj G [0, R], Ylj=i^j — ^o> 
k k / k \ k 

£/&) ^ E - A E 3 - = E - At ?) + ^ 
j=i j=i \j=i j j=i 

> k(f(t )-Xtl)+k\tl = kf(t ). □ 

We apply Lemma 17.41 to the function f(x) = $(— T + x). Let = 
stand for the standard Gaussian pdf. 

Lemma 7.5 Let f(t) = $(-T + t). Suppose 

2 _ _ / t r 



t „>0, T> ( „ + -, T<«<l— ^— ) . (7.4) 



Take A = 4>{—T + £ )/2io- 27ien t/ie assumptions of Lemma \7\4\ are fulfilled, i.e., 

inf (/(f) - At 2 ) = /(to) - At 2 . 

Proof. Denote g(t) = $(-T + t) - At 2 . By the choice of A we have g'(t ) = 0. Let 
us consider the second derivative, 

g"(t) = (T — t)<P(-T + 1) - 2A = (T - t)<f>(-T + t) - <j>{-T + t )/t . 

Observe that the function — x(p(x) is positive for x < 0, increases for x G (— oo, — 1) 
and decreases for x G (—1, 1); lim^-co 0(x) = = 00(0), 

/(to) = (T - t - t^M-T + t ) > 0. 

Consequently, there exist two points such that t\ < to < t 2 < T, 

g "(ti) = g"{t 2 ) = 0, g"{t) < as t < ti and t > t 2 . 

The function g(t) is therefore convex on [ti, t 2 ], concave on (— oo, ti] and on [t 2 , oo), 
and to is the point of a local minimum of g(t). By the concavity, this yields that the 
global minimum of g(t) at t G [0, R] is achieved either at £ = to or & t the ends of the 
interval [0, R\. Therefore we only need to show that g(0) > g(t ) and g(R) > g(t ). 

In order to verify the first inequality, observe that g(0) > 0. Recalling the well 
known inequality: 

$(-y) < ~<f>{-y), > , 

y 

we get 

g(to) = H-T + t ) - t o 0(-T + t )/2 < <j>(-T + t ) ( — ^— - ^ < , 



T-t ) 2 



because T > t + 2t . 

The second inequality follows from the relation 



g{R) = H -T + fl) - ^tl^A > t,-^(-T + h) > () 

2t 2t 

in view of the assumption on R. □ 
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Remark 7.1 Observe that if < v < u < b and 

T = uT p , t = vT p , R = bT p , 
where T p is large enough, then assumptions (17.41) hold. 
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